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1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)Q Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) [3 Notice of References Cited (PTO-892) 

2) [Z| Notice of Drafts person's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) [~1 Interview Summary (PTO-41 3) 

Paper No(s)/Mail Date. . 

5) CD Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



US- Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 1 1 



Application/Control Number: 09/9 1 5,23 5 Page 2 

Art Unit: 2671 

DETAILED ACTION 
Claim Rejections - 35 USC §103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1, 5, 9, 13, 17-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Lee et al., "Model Based Face Reconstruction For Animation" in view of Wiskott et al., "Face 
Recognition by Elastic Bunch Graph Matching". 

a. Referring to claim 1, Lee discloses providing a neutral-face front head image and 
a side head image for generating an avatar (". . .the face is in neutral expression. . ." - page 
5, paragraph 1; Fig 1.); automatically positioning nodes within feature locations on the 
front head image and the side head image (".. .provide automatic feature points extraction 
method. ..we consider. . .points such as eyes, nose, lip eyebrows and ears as feature 
points. . - page 6, paragraph 1; Fig. 1; Fig. 2); manually reviewing and correcting the 
node positions to remove artifacts and minimize distorted features in the avatar generated 
based on the node positions (". . .interface for interactive correction if and when 
needed..." - page 6, paragraph 1; Fig. 1). Lee does not explicitly disclose automatically 
finding head feature locations on the front head image and the side head image using 
elastic bunch graph matching. Wiskott discloses automatically finding head feature 
locations on a front head image and a side head image using elastic bunch graph 
matching ("graph. . .representing a face consists of nodes. . .the nodes are located at facial 
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landmarks. . graphs for new images can be generated automatically by elastic bunch 
graph matching..." - page 130, column 1, full paragraph 2; page 131, column 1; Fig. 1). 
At the time the invention was made, it would have been obvious to a person of ordinary 
skill in the art to modify the method of Lee by automatically finding head feature 
locations using elastic bunch graph matching. The suggestion/motivation for doing so 
would have been because using elastic bunch graph matching requires a general 
representation rather than models of individual faces (Wiskott, page 130, column 1, third 
full paragraph - column 2 first paragraph). 

b. Claim 5 is rejected with the rationale of claim 1. Claim 5 is claim 1 recited as a 
system. The modified method of Lee as applied to claim 1 above discloses a system 
("...we have developed a face recognition system. . ." - Lee, page 8, paragraph 5) 

c. Referring to claim 9, Lee discloses providing a neutral-face front head image and 
a side head image for generating an avatar (". . .the face is in neutral expression. . ." - page 
5, paragraph 1; Fig 1.); automatically positioning nodes at feature locations on the front 
head image and the side head image ("...provide automatic feature points extraction 
method. . . we consider. . .points such as eyes, nose, lip eyebrows and ears as feature 
points. . - page 6, paragraph 1; Fig. 1); manually reviewing and correcting the node 
positions to remove artifacts and minimize distorted features in the avatar generated 
based on the node positions (". . . interface for interactive correction if and when 
needed..." - page 6, paragraph 1; Fig. 1). Lee does not explicitly disclose automatically 
finding head feature locations on the front head image and the side head image using 
image analysis based on wavelet component values generated from wavelet 
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transformations of the respective neutral-face front head image and the side head image. 
Wiskott discloses automatically finding head feature locations on the front head image 
and the side head image using image analysis based on wavelet component values 
generated from wavelet transformations of the respective neutral-face front head image 
and the side head image (". . . a jet is based on a wavelet transform. . . a jet is defined as the 
set. . .of 40 complex Gabor wavelet coefficients. . .the. . FBG serves as a general 
representation of faces. Each stack of discs represent a jet. ..." - page 129, column 2, 
paragraph 2; Figure 2, caption). At the time the invention was made, it would have been 
obvious to a person of ordinary skill in the art to modify the method of Lee by 
automatically finding head feature locations using image analysis based on wavelet 
component values generated from wavelet transformations as taught by Wiskott. The 
suggestion/motivation for doing so would have been because using elastic bunch graph 
matching requires a general representation rather than models of individual faces 
(Wiskott, page 130, column 1, third full paragraph - column 2 first paragraph), 
d. Referring to claim 13, Lee does not explicitly disclose the wavelet 
transformations use Gabor wavelets. Wiskott discloses the wavelet transformations use 
Gabor wavelets (". . . faces are represented. . . by a Gabor wavelet transform. . . a jet is 
defined as the set. . .of 40 complex Gabor wavelet coefficients. . ." — Abstract page 129, 
column 2, paragraph 2). At the time the invention was made, it would have been obvious 
to a person of ordinary skill in the art to modify the method of Lee having the wavelet 
transformations use Gabor wavelets as taught by Wiskott. The suggestion/motivation for 
doing so would have been because using elastic bunch graph matching requires a general 
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representation rather than models of individual faces (Wiskott, page 130, column 1, third 
full paragraph - column 2 first paragraph). 

e. Referring to claim 17, Lee discloses creating the avatar based on geometric 
information associated with the automatically found head feature locations (page 7, 
paragraphs 2 - page 8, paragraph 1; Figs. 1 and 3). 

f Claim 18 is rejected per claim 9 with the rationale of the rejection of claim 17. 
d. Claim 19 is rejected per claim 5 with the rationale of the rejection of claim 17. 
3. Claims 2-4, 6-8, and 10-12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Lee in view of Wiskott as applied to claims 1, 5, and 9 above further in view of Thalmann 
et al., "Face to Virtual Face". 

a. Referring to claim 2, the modified method of Lee as applied to claim 1 above 
does not explicitly disclose applying an animation transform based on the corrected node 
positions for the neutral face. Thalmann discloses applying an animation transform based 
on the corrected node positions for the neutral face (". . .the mask can be interactively 
adjusted. . .to reproduce the corresponding movements on the virtual face, a mapping is 
carried out from the tracked features to the appropriate MPA's. . . for facial animation. . - 
— page 876, column 2, paragraph 1 - page 877, column 1, paragraph; Fig. 5). At the time 
the invention was made, it would have been obvious to a person of ordinary skill in the 
art to further modify the method of Lee by generating an animation transform based on 
the corrected node positions for the neutral face as taught by Thalmann. The 
suggestion/motivation for doing so would have been because it would allow the 
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mimicking of the facial expression of the real person on his clone (Thalmann, page 877, 
column 1, paragraph 1 - column 2, paragraph 1). 

c. Referring to claim 3, Lee does not explicitly disclose applying the animation 
transform to expression face avatar meshes for generating the avatar. Thalmann discloses 
applying the animation transform to expression face avatar meshes for generating the 
avatar (Figure 7). At the time the invention was made, it would have been obvious to a 
person of ordinary skill in the art to further modify the method of Lee by applying the 
animation transform to expression face avatar meshes for generating the avatar as taught 
by Thalmann. The suggestion/motivation for doing so would have been because it would 
allow the mimicking of the facial expression of the real person on his clone (Thalmann, 
page 877, column 1, paragraph 1 - column 2, paragraph 1). 

d. Referring to claim 4, Lee does not explicitly disclose applying the animation 
transform to morph targets. Thalmann discloses applying the animation transform to 
morph targets (". . .a mapping carried out from the tracked features to the appropriate 
MPA's. . . the allows us to mimic the facial expression of the real person. . - page 877, 
column 1, paragraph 1- column 2, paragraph 1; Fig. 7). 

e. Claims 6-8 are rejected per claim 5 with the rationale of the rejection of claims 2- 
4 respectively. 

f Referring to claim 10, the modified method of Lee as applied to claim 9 above 
does not explicitly disclose applying an animation transform based on the corrected node 
positions for the neutral face. Thalmann discloses applying an animation transform based 
on the corrected node positions for the neutral face (". . .the mask can be interactively 
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adjusted. . .to reproduce the corresponding movements on the virtual face, a mapping is 
carried out from the tracked features to the appropriate MPA's. . .for facial animation. 
— page 876, column 2, paragraph 1 - page 877, column 1, paragraph; Fig. 5). At the time 
the invention was made, it would have been obvious to a person of ordinary skill in the 
art to further modify the method of Lee by generating an animation transform based on 
the corrected node positions for the neutral face as taught by Thalmann. The 
suggest ion/motivation for doing so would have been because it would allow the 
mimicking of the facial expression of the real person on his clone (Thalmann, page 877, 
column 1, paragraph 1 - column 2, paragraph 1). 

g. Referring to claim 1 1, Lee does not explicitly disclose applying the animation 
transform to expression face avatar meshes for generating the avatar. Thalmann discloses 
applying the animation transform to expression face avatar meshes for generating the 
avatar (Figure 7). At the time the invention was made, it would have been obvious to a 
person of ordinary skill in the art to further modify the method of Lee by applying the 
animation transform to expression face avatar meshes for generating the avatar as taught 
by Thalmann. The suggestion/motivation for doing so would have been because it would 
allow the mimicking of the facial expression of the real person on his clone (Thalmann, 
page 877, column 1, paragraph 1 - column 2, paragraph 1). 

h. Referring to claim 12, Lee does not explicitly disclose applying the animation 
transform to morph targets. Thalmann discloses applying the animation transform to 
morph targets (". . .a mapping carried out from the tracked features to the appropriate 
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MPA's. . .the allows us to mimic the facial expression of the real person. . - page 877, 
column 1, paragraph 1- column 2, paragraph 1; Fig. 7). 

4. Claims 14-16 rejected under 35 U.S.C. 103(a) as being unpatentable over Lee in view of 
Wiskott as applied to claims 1, 5, and 9 above further in view of Cootes et al., "View-Based 
Active Appearance Models" (Cootes). 

a. Referring to claim 14, Lee does not explicitly disclose mapping sensed facial 
features to corresponding avatar meshes using linear regression. Cootes suggests 
mapping sensed facial features to corresponding avatar meshes using linear regression 
(page 5, column 1, full paragraph 1; page 2, column 1, paragraph 7 and column 2, full 
paragraph 1 - and column 2, full paragraph 1). At the time the invention was made, it 
would have been obvious to a person of ordinary skill in the art to further modify the 
method of Lee by mapping sensed facial features to corresponding avatar meshes using 
linear regression as taught by Cootes. The suggestion/motivation for doing so would have 
been because it could rapidly match any of the individual models to a new image (page 1, 
column 2, full paragraph 1) and the approach would be useful for driving animated 
avatars (page 5, column 2, paragraph 3). 

b. Claim 15 is rejected per claim 9 with the rationale of the rejection of claim 14. 

c. Claim 16 is rejected per claim 5 with the rationale of the rejection of claim 14. 

Response to Arguments 

5. Applicant's arguments filed 12/3 1/03 have been fully considered but they are not 
persuasive. Applicant contends that the combination of Lee and Wiskott fail to teach 

". . . positioning nodes within feature locations. . . " The Examiner notes that Lee teaches the 
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"snakes" consist of nodes (Lee, Fig. 2 and page 6, paragraph 3). The Applicant argues that Lee 
teaches placing contours along the edges of feature locations, not within the said feature 
locations. However, while not conceding the Applicant's argument, the Examiner would like to 
point out that a contour along the edge of a feature location is still within the feature location. 
In addition, the nodes used to outline the hair feature are located within the hair (Fig. 2). 
Applicant also argues there is no motivation to combine Lee and Wiskott However, has noted in 
the previous Office Action elastic bunch graph matching requires a general representation rather 
than models of individual faces, which would avoid the expense of constructing a graph for each 
feature combination (Lee, page 130, column 1, third full paragraph - column 2, paragraph 1) and 
note that Lee suggests integrating facial recognition methods (page 10, paragraph 6). Regarding 
the Applicant's contention that the methods of Wiskott may not be accurate enough for the 
purpose of creating an avatar, the Examiner notes that Lee discloses reconstruction of shape may 
not require high accuracy (page 4, full paragraph 3). Regarding the Applicant's argument that 
Thalmann uses MP As, not keyframes the Examiner again reminds the Applicant that although 
the claims are interpreted in light of the specification, limitations from the specification are not 
read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 
Regarding the Applicant's contention that Thalmann discloses animation of the model, not 
creation of the animation data the Applicant is again reminds the Applicant that although the 
claims are interpreted in light of the specification, limitations from the specification are not read 
into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). Note 
that the claim explicitly discloses ". . .applying the animation transform to expression face avatar 
meshes for generating the avatar." A close reading of the Thalmann reference would have 
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revealed that the animation transform is applied to expression face avatar meshes (Fig. 5, note 
the step in which the generic model is deformed and the subsequent generation of avatar). 
Regarding the change in Power of Attorney, the Examiner has checked the case file as requested 
and notes that the correct address is on file. Regarding the correction to title as printed on the file 
wrapper and official filing receipt, the appropriate changes have been made per MPEP 719.02. 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Linzy McCartney whose telephone number is (703) 605-0745. 
The examiner can normally be reached on Mon-Friday (8:00 AM-5: 30PM). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman, can be reached at (703) 305-9798. 
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Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service Office whose 
telephone number is (703) 306-0377. 
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